A digital image sensor is an integral part of several electronic systems including computers, cell phones, digital photography systems, robotics vision systems, security cameras, medical instruments, color matching applications and other color photosensitive devices. One digital image sensor, such as a TCS230 Programmable Color Light-to-Frequency Converter manufactured by TAOS, Inc., typically includes sensing elements, which may be sensitive to a broad range of frequencies of light. Some systems include lenses that are added and positioned on top of each sensing element to collect light and to make the sensing elements more sensitive.
Adding color filters to light sensors on an image sensor to specifically be color sensing elements may capture the brightness of the light that passes through and provide color sensing for red, green, and blue, for example. Thus, with color filters in place, color sensing elements typically record only the brightness of the light that matches its filler and passes through it while other colors are blocked. For example, a pixel with a red filter senses the brightness of only the red light that strikes it.
The color sensing elements appear in a photodiode grid consisting of groups of individual color sensing elements, each checkered throughout the grid on the same optical plane. For example, a group may include individual color sensing elements, for example, a red sensing element, a green sensing element, a blue sensing element, and sometimes a clear sensing element with no filter for intensity information. All of the photodiodes of the same color are typically connected in parallel. With the TCS230, the color selected for use during operation is dynamically selectable via two programming pins. If the output is digital, the output for individual color sensing elements is typically a square wave whose frequency is directly proportional to the intensity of the selected color.
As common as digital image sensors are to electronic systems, problems of defect sensitivity, spatial error, inference error and interpolation calculations make these benefits difficult to realize. Defect sensitivity is introduced in the fabrication of the sensing array in the conventional approach. Spatial error occurs because readings are taken at different points than the actual point interpolated; inference error takes place because each reading is not a direct observation. And, interpolation calculations involve a great deal of two-dimensional computation to be performed on each image in real time after each exposure through mathematical algorithms used to determine an appropriate color to assign to pixels in an image. In particular, algorithms typically interpolate visual colors at various points on a grid using a checkering of red, green, and blue sensing elements. Such interpolation computation requires sophisticated processing capability to be built into the imaging device. These interpolations translate into increased hardware cost, increased energy consumption, and slower cycle time for the image capturing device.
Unfortunately, this interpolation typically also introduces artifacts into the image resulting from the mathematical interpolation. An artifact is a distortion of the image that degrades image quality, for example, stair steps on a diagonal line.
One solution to defect sensitivity is to use firmware to ignore the information coming from a single sensing element of an array of sensing elements that has been found to be defective after its manufacture, and replace that information with additional interpolation from nearby elements. This solution, however, degrades image quality.
One solution to spatial error, inference error and interpolation has been making digital image sensors with a technology called X3, where the sensing elements are stacked vertically to read light at the same time. The colors, however, tend to appear undersaturated and cannot always be tuned to brilliance with software. In addition, vertically stacked sensing elements produce excessive noise, especially in shadows and red hues. The noise problem becomes even more severe at higher International Standards Organization (ISO) settings for photographic sensitivity; for example, ISO 400 is a well-known photographic sensitivity in the art of photography and typically indicates a digital emulation of a resulting traditional exposure based on a given shutter speed and aperture size. ISO 400 shots taken with X3 technology involved show multicolored noise that would ruin many prints. As the X3 sensor does not use a conventional checkerboard array of elements sensitive to a single color, use of the X3 sensor requires redesigning the camera system to accomodate the X3 sensor; such redesign increases system design costs and lengthens product development cycles. Furthermore, the X3 sensor outputs raw sensor data, requiring additional processing outside the camera, which is time-consuming and inconvenient.
Therefore, there is a need for methods and arrangements capable of capturing quality images with less spatial error, inference error and interpolation.